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Many of the achievement measures usually used In educational evaluation 
do v;t seem to provide enough data about hov aducational programs effect 
pupils 1 development on the more complex intellectual skills such as productive 
thin' ing and evaluation. Most measures which have been developed and 
standarized for higher cognitive skills are useable only with teenagers 
or adults. For the evaluator of elementary education programs, there is a 
dearth of instruments available to him. Prior to the inception of this study 
a particular need was felt by the evaluators of an individualized elementary 
school program to show whether or not L e program effects abilities related to 
verbal creativity. Using the structure of intellect model as a guide, seven 
tests were selected which had been shown in the past to be relatively factorially 
pure measures for abilities hypothesized to contribute to creativity in 
the semantic, area. A pilot study was conducted to see if these tests could 
be adapted to the upper elementary school level, grades four through six, 
and maintain their factorial validity. That pilot study is the subject of 
this report. 
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Structure of intellect factors selected for the study include the 
following: 

CMI - Cognition of Semantic Implications (conceptual foresight). 

This factor represents the ability to foresee the possible 
consequences or implications of a problem and/or its solu- 
tions . 



DMU - Divergent Production of Semantic Units (ideational fluency). 

Tests for this factor demonstrate ability to generate a large 
number of ideas about a certain problem. 

DMR - Divergent Production of Semantic Relations (associational 
fluency). This may be an ability to make meaningful con- 
nections between ideas by association thus leading to pos- 
sible solutions to a problem. 

DMT - Divergent Production of Semantic Transformations (original- 
ity). Tests for this factor seem to measure quality of re- 
sponses rather than quantity as is implied by DMU , The re- 
sponses sought are remote or clever and involve making unusu- 
al changes in ideas. 

EMI - Evaluation of Semantic Implications (sensitivity to prob- 
lems)* This represents the essential first step in crea- 
tive problem solving: that Is, the ability to see the need 

for a solution, 



o 
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Tests for these factors were adapted from those used in two previous 
studies by Merrifield, et^aT. (1964) and„ Schmadel, et. al_. (1965) with sixth 
and seventh grade children. These tests are as follows: 

1. ’Mays To Do It ir measures CMI, ability to see all th? elements 
involved in a problem and to see solutions that will meet the 
problem requirements. Each item in this test presents a job 
to be done and the pupil must tell as many ways as he can to 
do it. Sample Question: You have a deck of playing cards 
with the picture of a dog on the back. How many ways can you 
sort the cards, so that all the cards you put in the same 
group have something alike? 

2. "Similar Meanings” measures DMR, ability to produce related 
ideas. It asks the child to think of many different words 
which mean almost the same thing as a given word. Sample 
Question; What words mean almost the same as GOOD? 

3 & 4. ’Vhat Would Happen” gives the child a pretend change in the 
world and asks him to imagine other changes that it would 
cause. The test is scored in two ways to measure two different 
abilities. More obvious changes are counted to show ability 
to produce many simple ideas, DKU. Remote or way out changes 
are counted to show how well the child can think up unique or 
clever changes in ideas, DMT. Sample Question: What would 

happen if no one needed or wanted sleep? 
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5. ,r Seeing Problems," as Its title indicates, measures EMI, 

ability to see problems connected with an object. The 

£ 

name of an object is given and the child is asked to 
write down problems in using or taking care of it. 

Sample Question: What are some problems with a candle? 

6 & 7. "Names for Stories" asks the child to make up titles for 
a short story. This test is also scored in two ways. 
Clever titles indicate DMT, ability to think of unique 
changes in ideas. Not-so-clever titles are counted to 
indicate DMU, ability to think up many simple ideas. 
Sample Question: There was a man who could not hear his 

wife talking. She got him a hearing aid. He kept it 
turned on for a while, but then decided she talked too 
much. So he wore his hearing aid, but kept it turned 
off. Write titles for the story. 
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For administration, the tests ware grouped into two booklets to be 
used in two separate sessions. Each session contained about thirty 
minutes of testing with additional time allowed between tests for giving 
directions. There were three separately timed parts in each test, and 
a full page was devoted to each part, giving pupils adequate space for 
writing answers. Together, these tests were titled Measures of Childrens 
Abilities : Productive Thinking — Series M. They are copyrighted by Sheridan 

Psychological Services, Beverly Hills, California. At the end of the second 
booklet a short test of writing speed, in which the children were given one 
minute to write the alphabet as Many times as they could, was Included. 
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The tests were administered by specially trained testers to children in 
grades three through six on two consecutive days in early March, 1969. 
Approximately 1,300 pupils were tested in six schools, three individualized 
schools and their respective controls. The three pairs of schools represented 
three school districts in the North Eastern United States reflecting various 
degrees of urbanization . District one was a relatively affluent, white, 
semi-rural system. The schools from the second district represented the pri- 
marily white outskirts of a city with a large ethnic group population. District 
three schools were from the inner city and had mostly black students. 

Scoring of these open-ended tests was done by ten trained clerks with 
high school educations. Inter-rater reliabilities for the twenty-one part 
scores of the Productive Thinking battery ranged from .63 to .99 with eighteen 
of the twenty-one correlations above .80. 

Twenty- seven variables were included in the analysis of the data. 

These were treatment (individualized instruction versus the control), school 
district, grade, sex, verbal I.Q., writing speed, and tne twenty-one part scores 
on the Productive Thinking battery. Since I.Q. scores were not available 
for the third graders, only children in grades 4, 5, and 6 with complete data 
records, a total of 893 cases, were used in the analysis. The first step in 
the analysis was to obtain frequency distributions and descriptive statistics 
on each variable. Skewed distributions were then adjusted to make them 
symmetrical. Next, intercorrelations of the twenty-seven variables were 
obtained, and the correlations were corrected for course grouping. A factor 
analysis of the corrected correlation matrix was done using the Biomedical 
Computer Program, BMD-03M. In this analysis, the maximum absolute row values 




analysis would yield five factors of productive thinking (see page 2 for 
description of hypothesized factors), it was judged advisable to consider 
six principal components to allow for the socio-economic, verbal I.Q., and 
grade influences. The BMD-03M program requires the specification of a 
minimum eigenvalue for the factors to be rotated; an eigenvalue of 0.5000 
was chosen, and six principal components having eigenvalues greater than that 
number obtained. These six were then rotated to the varimax criterion. After 
factor analysis of the total group, the study population was split by grade 
and school district and nine separate factor analyses were done for the nine 
sub-samples thus created. Factor structures (varimax criterion) of the nine 
sub-samples were compared with that of the total sample. 

The Total Sample Factor Matri x 

When the intercorrelations of the twenty-seven variables for the total 
sample were factored, six principal factors appeared to exhaust most of the 
common variance; the sum of the six eigenvalues for these factors was equal 
to 94% of the sum of the comrounality estimates in the correlation matrix. 

These six factors were rotated to the varimax criterion. The rotated factor 
matrix is presented in Table I. 



Factor A was interpreted to represent the effect of maturation or increasing 




level of grade in school on pupils 1 scores. The factor contained significant 
loadings for the following variables: 
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Grade 



.62 



Writing Speed 



Ways To Do It, Part 1 



.48 



What Would Happen (remote) Part 2 



.40 



What Would Happen (remote) Part 1 



.38 



Ways To Do It, Part 3 



.37 



Seeing Problems, Part 3 



.30 



Seeing Problems, Part 2 



.30 



SCK 



.30 



Ways To Do It, Part 2 



.29 



Of the ability variables loading on this factor, five of them ware from tests 
hypothesized for this study to measure abilities to deal with implications. 

Ways To Do it was hypothesized as a test for cognition of semantic impli- 
cations, CMI, and Seeing Problems was thought to be a test for EMI, evaluation 
of semantic implications. What Would Happen seems logically to require 
examinees to produce implications; therefore, it is not surprising that two parts 
of What VIould Happen (remote) exhibited common variance with the Ways To Do It 
and Seeing Problems tests. Perhaps Factor A represents an ability to produce 
implications which is related to maturation in this age group; this would be 
consistent with positive loadings of grade level and proficiency at writing 
speed. Viewed this way, Factor A might also have been interpreted to represent 
Divergent Production of Semantic Implications, DMI, 

Factor B seemed to represent the ability Divergent Production of Semantic 
Units, DMU, often referred to as ideational fluency. It included significant 
Vikings f° r th e following variables: 
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Names For Stories 


(low) 


Part 


2 


.80 


Names For Stories 


(low) 


Part 


3 


.78 


Names For Stories 


(low) 


Part 


1 


.74 


Seeing Problems 




Part 


1 


.38 


Seeing Problems 




Part 


3 


.35 


Seeing Problems 




Part 


2 


.32 


Sex 








.27 


What Would Happen 


(obvious) Part 2 


.26 



Names for Stories (low) was hypothesized to measure this factor at the outset 
of the study. The fact that parts of three different tests load on this factor 
is encouraging in that it suggests that the factor does not represent a single 
test but a mental ability that goes beyond a single measuring instrument. 
However, the case for test specific variance could be made. 

For factor C, familiar tests from previous studies (Merrifield, j^t. al . , 
1964; and Schmadel, e_t. al. 1965) suggested its definition. It was interpreted 
as Divergent Production of Semantic Trans f* rmations, DMT, also referred to as 
originality. Variables loading significantly on factor C were: 

Names For Stories (high) Part 3 .72 

Names For Stories (high) Part 2 .70 

Names For Stories (high) Part 1 .60 

What Would Happen (remote) Part 1 .42 

Seeing Problems Part 1 .36 

Seeing Problems Part 2 .33 

What Would Happen (remote) Part 2 .29 
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Grade 




.28 


What Would Happen 


(obvious) Part 2 


.28 


Ways To Do It 


Part 2 


.27 



Natnes for Stories (high) and What Would Happen (remote) were the hypothesized 
tests for this factor. The evidence of the contribution of Divergent Production 
ability or originality to the other tests loading on this factor is logically 
acceptable. As in factor B, there is the possibility that specific and common 
variance are confounded in this result. 

Factor D contains the following significant loadings: 



School District 
Lorge-Thotndike Verbal 1.0. 




-.71 

.65 


Ways To Do It 


Part 2 




.52 


Seeing Problems 


Part 3 




.49 


Seeing Problems 


Part 2 




.49 


Seeing Problems 


Part 3 




.45 


What Would Happen 


(remote) Part 


1 


.45 


What Would Happen 


(obvious) Part 


2 


.36 


What Would Happen 


(remote) Part 


2 


.34 


What Would Happen 


(obvious) Part 


3 


.30 


Similar Meanings 


Part 3 




.29 


Names For Stories 


'high) Part 1 




.29 



This factor was defined primarily by Lorge-Thorndike Verbal I.Q. and school 
district with other rigniftcant loadings for parts from six of the seven tests 
in the Productive Thinking battery. One of them, Seeing Problems, hypothesized 

as a measure of EMI, had it^ highest loadings for all three parts on this 

0 
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factor, which is not lnconsri stent with the strong dependence of the usual 
Verbal-I.Q. measures on processes of cognition and evaluation. The other 
factors (A, B, and C) on Which Seeing Problems had significant loadings 
indicate its task requirement of productive thinking processes showing it 
to be a rather complex test. 

Factor D seems to represent some so'rt of generalized language or reading 
ability effecting performance on these semantic tests-**an ability possessed 
in greater measure by pupils in the higher socio-economic school district 
(a ; lower numerical code denotes higher SES, thus the negative loading for 
school district). The pervasiveness of language skill or reading ability in 
the performances elicited by this battery of tests was evident in factor D. 

In the analysis, the use of data from a sample pooled over three grade levels 
and different socio-economic strata enhanced the opportunity for systematic 
covariance with language or reading skills to emerge. It is logically under- 
standable that such language knowledge as measured by the Lorge-Thorndike 
Tests would facilitate performance on these semantic tests. However, ic is 
apparent that systematic variation in this kind of language facility did not 
account for the systematic variability in performance evidenced by the remain- 
ing factors which resulted from this analyst^. Other factors were Independent 
of community and the kind of culturally biased language measure provided by 
the verbal T.Q. test. 

The productive thinking Vc.ilables which did relate to factor D should be 
noted. Out of ten loadings, eight were for parts of Ways To ^ It, Seeing 
Problems, and What Would Happen (obvious and remote). These tests, as discussed 
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in connection with factor A, may measure some sort of ability to work with 
implications. Factor £ seems to suggest that facility with implications is 

A 

developing in intermediate grade children, and factor D seems to sug 6 ast that 
proficiency with language enhances that development. A plot of A versus D 
shows that if one disregards variables 2 and 3 {school district and grade), 
which are independent by the design of the study, the two bounding hyperplanes 
have normals on one of which the variables of Sex, Writing Speed, and Ways To 
Do It, Part 1 have significant projections, while I.Q., Ways To Do It, Part 2, 
and all 3 parts of Seeing Problems dominate the other. The latter is a language 
implications composite. The two oblique hyperplanes are separated by approxi- 
mately 40°, and their normals by 140°. This type of result is not found in 
the nine subsamples, in which variables 2 and 3 have no variability. 

Factor E represented another ability independent of language and 
community. It is defined entirely by the three parts of the Similar Meanings 
test : 



Similar Meanings 


Part 2 


.62 


Similar Meanings 


Part 1 


.57 


Similar Meanings 


Part 3 


.56 



It was considered to represent DMR, Divergent Production of Semantic Relations, 
uS hypothesized. It is possible that some test-specific variance is being 
confounded with common factor variances here. 

The only test variables loading on Factor F arc the obvious and remote 
scores for What Would Happen, Part 3. (Ther^ was also a barely significant 
positive loading for sex.) The loadings wore as follows: 

What Would Happen (remote) Part 3 .51 

11 
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Whac Would Happen (obvious) Part 3 -.46 

Sex 9 .27 

This factor is probably an artifact resulting from a situation whers most 
examinees made nearly the same low number of responses to the question. 

When this happens , the mutually exclusive scoring procedure may induce a 
negative correlation. The negative correlation embedded in a generally low 
or positive matrix results in its "own factor." This seems to be the case 
with Factor F. 

To summarize the factor structure for the total sample, six principal 
factors were rotated and interpreted. Three (factors B, C, and E) appeared 
to be hypothesized structure of intellect factors in the area of Divergent 
Production. They were DMR (Divergent Production of Semantic Relations), 

DMU (Divergent Production of Semantic Units), and DMT (Divergent Production 
of Semantic Transformations). These factors were independent of verbal I.Q. 
and school district or grade. A fourth factor (f ictor D) seemed to represent 
a generalized language facility defined by the verbal I.Q. measure and strongly 
related to community. The fifth factor (factor A) was interpreted as primarily 
reflecting the effect of grade in school or maturation. Both the I.Q. factor 
(factor D) and maturation factor (factor A) contained loadings for measures 
of ability hypothesized to deal with semantic implications (CMI 6 EMI) which 
did not appear as separate factors. The sixth factor (factor F) appears to 
be merely an artifact of the scoring procedure used and was defined by the 
two scores from only one test part. 
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The Sub-Sample Factor Matrices 

Because of the large amounts of variance in factors A and D accounted for 

ij 

by the variables of grade and school district respectively, it was decided to 
stratify the sample on these two variables to see if their effects could be 
limited, thus allowing other intellectual factors to show up more clearly in 
the factor structure. Nine sub-samples — fourth, fifth and sixth grade groups 
In each of three school districts — were created. Correlation matrices without 
the variables of grade and school district were generated and factored by the 
same procedure described for the u\al sample. Perusal of the nine resultant 
factor structures revealed many similarities among them and with the total 
sample factor matrix. 

I.Q. loaded on factors with the experimental tests in five of the 
nine sub-samples. The tests effected by I.Q, were Seeing Problems, Ways To 
Do It, and What Would Happen. These instances occurred in all three fifth 
grades and in one fourth grade and one sixth grade group. Analyses of the 
rine sub-samples showed that sex was related to writing speed in two groups, 
and to isolated parts from the What Would Happen and Ways To Do It tests in 
four other groups, Sex was related to the Names for Stories (low) test in 
two groups, and in one of those all three parts of the test had high loadings 
on this sex factor. As with I.Q., factor loadings for sex did not occur in 
any one particular grade or school district. 

One of the strongest of the experimental tests seemed to be the Similar 
Meanings test for factor DMR, Divergent Production of Semantic Relations. 

This test occurred as a unique factor in all nine sub-samples. In one fourth 
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grade sample, it vas related to writing speed, also. In four of the 
sub-samples one part from the Similar Meanings test had a noticably weaker 
factor loading than did the others. 

Another strong factor test was Names for Stories (low) measuring DMU, 
Divergent Production of Semantic Units. This factor occurred in eight of 
the nine sub-samples with all parts having factor loadings significant at 
the .05 level. In one sixth grade group, sex was also strongly related to 
this factor with the girls doing better. 

Names for Stories (high) occurred as a unique factor in five of the 
nine sub-samples. These were all on the fifth and sixth grade levels and 
represented all three of the school districts. The test measures DMT, 
Divergent Production of Semantic Transformations. In two fourth grade 
samples, parts 2 and 3 of the test loaded on this factor but part one did 
not, and in one fourth grade sample girls did better. 

In one fourth grade sample, Seeing Problems emerged as a unique factor 
test. However, it is doubtful that it was measuring only the hypothesized 
factor. Evaluation of Semantic Implications (EMI), since the test parts 
loaded rather consistently with I.Q* in the other analyses, and in one sub- 
sample with DMU, Divergent Production of Semantic Units. 

The nine sub-analyses do not seem to change greatly the picture obtained 
from the factor analysis of the entire group; factor structures for each grade 
and school district are similar. The confusion among the Ways To Do It, 
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What Would Happen, and Seeing Problems tests and I.Q. remains. All three of 
these aptitude tests seem to have something to do with the recognition and 
production of implications. If skill in h the cognition of implications is 
underlying performance on these three tests, this could also help explain the 
relationship with I.Q., since intelligence tests generally contain measures 
of cognition; however, the hypothesized measure for CMI, Cognition of Semantic 
Implications had a poor showing in this study. The factors DMU, DMR, and DMT 
which did appear consisently and seemed to be unrelated to I.Q., school district, 
or age were all divergent production factors, which relate to the ability to 
think up many different ideas about one situation or idea. Names for Stories 
seems a useful measure for both DMU and DMT, relatively uninvolved with I.Q. 
Similar Meanings is a dependable measure for DMR. However, What Would Happen 
is disappointing, relative to its dependability in older examinees; similarly, 
Ways To Do It seems ineffective for this age level, and the question of whether 
CMI can be differentiated in such examinees remains unanswered. Seeing Problems, 
despite its involvement with verbal I.Q., provides an interesting and internally 
consistent measure of a relative dis tinct, though complex, ability . 

Conclusion 

Two overall results of this study are noteworthy. One is the separation 
among three divergent production factors, and an I.Q. — socio-economic factor, 
Indicating that divergent production tests may show promise as ability measures 
which are more nearly fair in studies Including both black and white children. 

The second noteworthy result is the confusion among tests apparently measuring 
ability to work with implications. These results suggest that seeing impli- 
cations — perhaps a kind of sensitivity to problems — is a broad, rather fluid 

ability at this age level and brandies into structure-of-Intellcct categories 

O 
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which may not appear as unique abilities until later ages. 

The implication from this study is that, should one wish to emphasize 

* 

divergent production in elementary education, some tests for measuring such 
abilities do exist and can be developed further for the evaluation of such an 
emphasis. Further test development along these lines is in progress, including 
figural and symbolic abilities as well as those in the semantic area. 
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TABLE I 



ROTATED FACTOR MATRIX FOR SAMPLE OF COMBINED SCHOOLS AND GRADES (893 cases! 



VARIABLE 


FACTORS 








A 


B 


c 


D 


E 


F 








Divergent 


Divergent 


Verbal i.q. 


Divergent 










Production of 


Production 


and Socio- 


Production 










Semantic 


of Semantic 


Economic 


of Semantic 








Maturation 


Units (DMU) 


Transforma- 


' Status 


Relations 








Effect 




tions (DMTI 




IDMR) 


Artifact 


1. T reatment 




-0.01069 


0.01888 


0.04601 


-0.19711 


-0.11123 


C.00G90 


2. School District 




-0.02102 


0.03545 


0.17411 


-0.70878 


-009255 


0.01132 


3. Grade 




0.62096 


0.10191 


0.28228 


0.01048 


0.13116 


0.05187 


4. Sex 




-0.29789 


-027419 


0.21490 


-0.074S2 


•0.08940 


0.272(30 


5. Verbal 1. Q. 




0.24274 


0.06626 


0.15562 


0.64800 


0,12497 


0.03707 


6. Writing Speed 




0,51374 


0.10758 


0.07549 


0,16941 


0.23707 


-0.16163 


7. Ways To Do It 


?&rt 1 


0.48472 


0.12332 


0.15304 


0.25725 


0.05538 


0.06401 


8. Ways To Do 1 1 


Part 2 


0.29197 


0.13119 


0.27281 


0.51856 


0.06600 


-0.04336 


9. Ways To Do 1 1 


Part 3 


0.36629 


0.03366 


0.05456 


0.20983 


0.1 1985 


-0.02584 


10. Similar Meanings 


Part 1 


0.22919 


0.17382 


0.22107 


0.15052 


0.57255 


0.03574 


1 1. Similar Meanings 


Part 2 


0.14364 


0.24547 


0.18766 


0.11197 


0.62216 


0.02009 


12. Similar Meanings 


Pari 3 


0.18-643 


0.19723 


0,18285 


0 23721 


0.56279 


005853 


13. What Would Happen (obvious) 


Pan 1 


0.04642 


0.24582 


0.07496 


0.19249 


0.23682 


-0.22333 


14. What Would Happen (obvious) 


Part 2 


0.17813 


0.26466 


0,28163 


0.35652 


0.15313 


-032824 


1 5. What Would Happen (obvious) 


Part 3 


0.21342 


0.23018 J 


0 23026 


0.30198 


U. 11426 


-0.46583 


16. What Would Happen (remote) 


Part 1 


0.37655 


0.15793 ' 


041590 


0.44903 


006795 


0.19703 


1 7. What Would Happen (remote) 


Part 2 


0.40247 


0.17519 


0.28862 


0.34333 


0.03 ICO 


0.22573 


1 6, What Would Happen (remote) 


Part 3 


0.12754 


0.03921 


0.17031 


0.19938 


0.17240 


0.51211 


19. Seeing Problems 


Part 1 


0.22538 


0.3/631 


0.35787 


0.45117 


015770 


-0.01334 


20. Seeing Problems 


Part 2 


0.29910 


0.32524 


0.32750 


0.49154 


0.1663& 


-0.01372 


21. Seeing Problems 


Part 3 


0.30265 


0.34650 


023892 


0.49260 


0.11028 


-0.03132 


22. Names For Stories flow) 


Part 1 


0.03472 


0.74254 


0.05283 


0.03584 


0.09224 


-0.07133 


23. Names For Stories (low) 


Part 2 


0.14063 


0.80387 


0.06540 1 


0.11009 


0.15236 


0.02331 


24. Names For Stories (low) 


Part 3 


0.07G&4 


0.77769 


—0 12474 


0.01380 


0.19788 


0.03225 


25. Names For Stones (high) 


Part 1 


0 . 2 am 


0.13331 


0,59533 


0.28606 


0.229 00 


0, 12957 


26. Names For Stories (high) 


Part 2 


0.0792? 


-0.08462 


0.70124 


0.19017 


020103 


-0.00290 


27. Narr^s For Stories (high) 


Ptrt 3 


0.21790 


“0.01794 


0.71552 


0.10G45 


0.10976 


0.0004S 



NOTE: A lac tor load / no ol .?G1 is significant at (he .05 level. 



ERiC 



18 



